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AUG  2  5  2005 

MEMORANDUM  FOR  AUDITOR  GENERAL,  DEPARTMENT  OF  THE  ARMY 

SUBJECT:  Quality  Control  Review  of  Army  Audit  Agency’s  Special  Access  Program 
Audits  (Report  No.  D-2005-6-008) 

We  are  providing  this  report  for  your  information  and  use.  We  reviewed  the  Army 
Audit  Agency  (AAA)  system  of  quality  control  over  Special  Access  Program  (SAP) 
audits  for  the  three  years  ended  September  30, 2004.  The  Government  Auditing 
Standards  (GAS)  require  that  an  audit  organization  performing  audits  and/or  attestation 
engagements  in  accordance  with  GAS  should  have  an  appropriate  internal  quality  control 
system  in  place  and  undergo  an  external  peer  review  at  least  once  every  three  years  by 
reviewers  independent  of  the  audit  organization  being  reviewed.  As  the  organization  that 
has  audit  policy  and  oversight  responsibilities  for  audits  in  the  DoD,  we  conducted  this 
external  peer  review  of  the  AAA  SAP  audits  in  conjunction  with  the  Air  Force  Audit 
Agency  (AFAA)  external  peer  review  of  AAA  non-SAP  audits. 

An  audit  organization’s  quality  control  policies  and  procedures  should  be 
appropriately  comprehensive  and  suitably  designed  to  provide  reasonable  assurance  of 
meeting  the  objectives  of  quality  control.  We  tested  the  AAA  SAP  system  of  quality 
control  to  the  extent  considered  appropriate. 

In  our  opinion,  the  system  of  quality  control  for  the  audit  function  of  AAA  SAP 
audits  in  effect  for  the  period  ended  September  30,  2004,  was  designed  in  accordance 
with  quality  standards  established  by  GAS.  Further,  the  internal  quality  control  system 
was  operating  effectively  to  provide  reasonable  assurance  that  SAP  audit  personnel  were 
following  established  policies,  procedures,  and  applicable  auditing  standards. 
Accordingly,  we  are  issuing  an  unqualified  opinion  on  your  SAP  audit  quality  control 
system  for  the  review  period  ended  September  30,  2004. 

Appendix  B  contains  comments,  observations,  and  recommendations  where  AAA 
can  improve  its  quality  control  program  related  to  SAP  audits,  as  well  as  our  responses  to 
AAA  management  comments  to  draft  and  revised  draft  versions  of  this  report. 

Appendix  C  provides  the  full  text  of  management  comments  in  response  to  both  the 
original  and  revised  draft  reports. 


See  Appendix  A  for  the  scope  and  methodology  of  the  review.  We  wish  to  express 
our  thanks  to  you  and  your  staff  for  your  cooperation  and  professionalism.  Please  contact 
Mr.  Robert  L.  Kienitz,  Acting  Project  Manager  at  (703)  604-8754  or  Ms.  Carolyn  R. 
Davis,  Deputy  Assistant  Inspector  General  for  Audit  Policy  and  Oversight  at 
(703)  604-8877  if  you  have  any  questions. 


Assistant  Inspector  General 
Audit  Policy  and  Oversight 


Cc:  Director,  DoD  Special  Access  Program  Coordination  Office 
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Appendix  A.  Scope  and  Methodology 


We  limited  our  review  to  the  adequacy  of  AAA  SAP  audits’  compliance  with  quality 
policies,  procedures,  and  standards.  We  judgmentally  selected  3  SAP  audits  from  a 
universe  of  72  formal  reports  issued  by  AAA  SAP  auditors  during  FYs  2002,  2003,  and 
2004,  and  tested  each  audit  for  compliance  with  the  AAA  system  of  quality  control.  The 
AFAA  conducted  a  review  of  the  AAA  internal  quality  control  system  for  non-SAP  audits 
and/or  attestation  engagements  and  has  issued  a  separate  report.  The  Assistant  Inspector 
General  for  Audit  Policy  and  Oversight  will  issue  an  overall  opinion  report  on  the  AAA 
internal  quality  control  system  that  will  include  the  combined  results  of  the  SAP  and 
non-SAP  reviews. 

In  performing  our  review,  we  considered  the  requirements  of  quality  control  standards 
and  other  auditing  standards  contained  in  the  2003  Revision  of  the  Government  Auditing 
Standards  issued  by  the  Comptroller  General  of  the  United  States.  GAS  3.52  states: 

The  external  peer  review  should  determine  whether,  during  the  period  under  review,  the 
reviewed  audit  organization’s  internal  quality  control  system  was  adequate  and  whether 
quality  control  policies  and  procedures  were  being  complied  with  to  provide  the  audit 
organization  with  reasonable  assurance  of  conforming  with  applicable  professional 
standards.  Audit  organizations  should  take  remedial,  corrective  actions  based  on  the 
results  of  the  peer  review. 

We  conducted  this  review  in  accordance  with  standards  and  guidelines  established  in  the 
Draft  2004  President’s  Council  on  Integrity  and  Efficiency  (PCIE)  “Guide  for  Conducting 
External  Peer  Reviews  of  the  Audit  Operations  of  Offices  of  Inspector  General.”  We 
modified  the  Guide  to  ensure  consistency  with  the  AFAA  review  of  non-SAP  audits,  and 
to  reflect  the  unique  nature  of  auditing  within  a  SAP  environment.  We  reviewed  audit 
documentation,  interviewed  AAA  auditors  and  their  managers,  reviewed  AAA  internal 
audit-related  policies  and  procedures.  We  performed  this  review  from  April  to  June  2005 
at  two  AAA  field  offices. 

We  used  the  following  criteria  to  select  the  audits  under  review: 

•  Worked  backward  starting  with  FY  2004  audits  in  order  to  review  the  most 
current  quality  assurance  procedures  in  place. 

•  Eliminated  Base  Realignment  and  Closure  audits  because  they  are  not 
considered  typical  audits. 

•  Avoided  audits  with  multiple  SAPs  associated  with  the  audit  for  ease  of 
access. 

•  Avoided  audits  that  have  the  same  or  similar  titles,  to  ensure  review  of 
multiple  types  of  projects. 
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The  following  table  identifies  the  specific  reports  reviewed. 


«  “31  » 

A-2004-0296- 

Coordinating  Requirements  for  Special  Access 

AMI 

Program,  Site  B 

A-2004-0276- 

AMI 

Department  of  the  Army  Support  Program 

A-2003-0386- 

06  August 

Intelligence  Support  to  Special  Access  Programs, 

AMI 

2003 

Site  E 

Limitations  of  Review.  Our  review  would  not  necessarily  disclose  all  weaknesses  in  the 
system  of  quality  control  or  all  instances  of  noncompliance  with  it  because  we  based  our 
review  on  selective  tests.  There  are  inherent  limitations  in  considering  the  potential 
effectiveness  of  any  quality  control  system.  In  performing  most  control  procedures, 
departures  can  result  from  misunderstanding  of  instructions,  mistakes  of  judgment, 
carelessness,  or  other  human  factors.  Projecting  any  evaluation  of  a  quality  control 
system  into  the  future  is  subject  to  the  risk  that  one  or  more  procedures  may  become 
inadequate  because  conditions  may  change  or  the  degree  of  compliance  with  procedures 
may  deteriorate. 
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Appendix  B.  Comments,  Observations,  and 

Recommendations 


We  are  issuing  an  unqualified  opinion  because  this  is  the  first  external  peer  review  for 
AAA  SAP  audits  and  the  issues  we  identified  were  not  cumulatively  significant  to  the 
reports’  findings,  conclusions,  and  recommendations.  Overall,  we  found  that  AAA  could 
improve  the  quality  control  program  and  guidance  related  to  the  areas  of  Supervision, 
Evidence  and  Audit  Documentation,  Reporting,  and  Quality  Assurance  for  SAP  audits. 
Implementing  the  recommendations  would  improve  the  quality  control  system  and  help 
maintain  an  unqualified  opinion. 

Supervision.  The  2003  revision  of  the  GAS  issued  by  the  Comptroller  General  of  the 
United  States  requires  staff  to  be  properly  supervised  and  reviews  of  audit  work  to  be 
documented.  Additionally,  AAA  Regulation  36-3,  “Audit  Survey  and  Execution,”  states 
that  supervisory  reviews  of  working  papers  are  mandatory  and  supervisors  must 
document  their  reviews. 

We  found  that  the  Auditors-In-Charge  (AIC)  usually  documented  their  reviews  of 
working  papers  with  initials  and  dates.  However,  we  found  that  most  working  papers 
created  by  the  AIC  did  not  have  documented  evidence  of  higher  level  supervisory  review. 

The  AAA  SAP  audit  environment  results  in  limitations  on  availability  of  auditors  that 
have  access  to  the  programs  because  of  the  special  security  requirements.  As  a  result, 
AICs  will  generally  conduct  more  field  work  than  in  an  unclassified  environment. 
Therefore,  in  such  an  environment,  supervision  of  AIC  working  papers  becomes  more 
important  as  a  quality  control  measure.  AAA  SAP  audit  managers  should  have 
procedures  in  place  to  ensure  they  satisfy  the  supervision  standard  at  all  levels  for 
working  papers  of  staff  auditors  and  AICs,  including  consideration  of  alternative  methods 
of  documenting  supervision  of  AICs. 

Recommendation.  We  recommend  that  the  Army  Auditor  General  develop 
alternative  policy  guidance  or  amend  AAA  Regulation  36-3  to  include  procedures  for 
documenting  supervisory  review  of  the  work  of  AICs  within  the  SAP  environment. 

Management  Comments.  The  Army  Auditor  General  partially  concurred  with 
the  recommendation,  stating  that  while  supervisory  reviews  must  be  performed  and 
documented,  alternative  policy  guidance  or  an  amendment  to  existing  guidance  is  not 
necessary.  The  Army  Auditor  General  stated  that  during  the  course  of  the  peer  review  of 
SAP  audits,  the  Program  Director  for  SAP  audits  issued  guidance  emphasizing  the 
requirement  for  supervisory  reviews.  In  addition,  the  Army  Auditor  General  stated  that 
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AAA  would  issue  a  memorandum  to  all  agency  personnel  discussing  the  results  of  the 
peer  review  and  reminding  them  of  the  existing  guidance  related  to  supervision.  AAA 
also  plans  to  include  supervisory  review  of  working  papers  as  a  special  interest  item 
during  its  future  post  audit  reviews  and  the  FY  2006  functional  reviews.  In  addition,  as 
part  of  a  new  AAA  regulation  on  the  report  writing  process  (to  be  issued  by 
September  30,  2005),  the  Independent  Review  Checklist  will  include  a  question  on 
whether  the  working  papers  have  been  reviewed. 

Reviewer  Response.  Management  comments  meet  the  intent  of  the 
recommendation.  The  AAA  memorandum  discussing  the  results  of  the  peer  review  and 
reminding  agency  personnel  of  existing  guidance  related  to  supervision  was  issued  on 
July  29,  2005. 

Evidence  and  Audit  Documentation.  The  GAS  requires  that  auditors  prepare  and 
maintain  audit  documentation  to  form  the  principal  record  of  the  audit  work  performed 
and  the  conclusions  reached.  AAA  Regulation  36-3  states  that  AAA  auditors  must  ensure 
the  evidence  in  audit  working  papers  is  adequate  to  support  the  audit  results,  to  include  all 
statements  of  fact  presented  in  the  audit  report.  In  addition,  AAA  Regulation  36-3  states 
that  as  a  general  rule  all  working  papers  or  groups  of  related  working  papers  should 
contain  certain  basic  information  such  as  purpose,  source,  scope,  results,  and  conclusions. 

Although  we  found  AAA  working  papers  contained  sufficient,  competent,  and  relevant 
evidence  to  support  judgments  and  conclusions  in  the  reports,  improvements  could  be 
made  to  audit  documentation.  Some  of  the  deficiencies  we  noted  during  our  review  were: 

•  Documentation  to  support  changes  to  facts  and  figures  between  the  draft  and  final 
report  was  lacking  in  1  of  the  3  reports  reviewed. 

•  Source  documents  were  missing  information  about  the  provider  of  such  documents 
for  1  of  the  3  reports  reviewed. 

•  Working  papers,  such  as  background  information  and  briefing  charts,  were  missing 
purpose,  source,  scope,  and  conclusion  for  all  3  of  the  reports  reviewed. 

Recommendation.  The  Army  Auditor  General  should  remind  all  SAP  audit 
personnel  to  comply  with  established  guidance  for  working  papers,  documentation,  and 
audit  evidence. 

Management  Comments.  The  Army  Auditor  General  concurred  with  the 
recommendation,  and  stated  that  AAA  would  issue  a  memorandum  to  all  agency 
personnel  discussing  the  results  of  the  peer  review  and  reminding  them  of  the  existing 
guidance  for  working  paper  preparation  and  content.  The  memorandum  would  also 
remind  agency  personnel  of  existing  requirements  relating  to  documenting  audit  work  and 
cross  referencing  the  work. 
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Reviewer  Response.  Management  comments  are  responsive.  The  AAA 
memorandum  discussing  the  results  of  the  peer  review  and  reminding  agency  personnel 
of  existing  guidance  for  working  paper  preparation  and  content  was  issued  on 
July  29,  2005. 

Reporting.  GAS  states  that  evidence  included  in  audit  reports  should  demonstrate  the 
correctness  and  reasonableness  of  the  matters  reported.  Furthermore,  GAS  states  that  one 
way  to  help  ensure  the  audit  report  is  accurate  is  to  use  a  quality  control  process  such  as 
independent  referencing,  a  process  in  which  an  experienced  auditor  who  is  independent  of 
the  audit  verifies  that  statements  of  fact,  figures,  and  dates  are  correctly  reported,  that 
findings  are  adequately  supported  by  the  audit  documentation,  and  that  the  conclusions 
and  recommendations  flow  logically  from  the  support. 

AAA  Regulation  36-85,  “Independent  Report  Referencing,”  requires  supervisors  to 
obtain  an  Independent  Reference  Review  prior  to  issuing  the  draft  report  to  Command. 
The  review  must  be  completed  by  a  senior  auditor  (GS-12  or  above)  not  associated  with 
the  audit  under  review  who  “verifies  the  accuracy  of  the  data  in  the  draft  audit  report  (and 
final  report,  if  changes  warrant)  by  tracing  the  data  to  supporting  documentation  in  the 
working  paper  files. . ..  In  cases  where  all  qualified  auditors  at  a  field  office  work  on  the 
audit  and  funding  constraints  prevent  assigning  an  outside  auditor  on  temporary  duty,  the 
level  2  supervisor  may  seek  a  waiver  in  order  to  use  auditors  on  the  same  audit  to 
reference  areas  they  did  not  work  on.” 

Additionally,  AAA  Regulation  36-85  states  that  “The  referencer  will  initial  each  fact  and 
figure  verified  in  the  report  to  help  ensure  they  overlook  nothing.  In  addition,  the 
referencer  will  initial  each  fact  and  figure  directly  on  the  supporting  working  paper 
evidencing  they  found  the  support.” 

We  found  that  AAA  SAP  audit  reports  generally  met  the  GAS  reporting  standards  for  the 
format  of  the  reports,  report  contents,  report  quality,  and  report  issuance  and  distribution. 
However,  for  two  of  the  three  SAP  audit  reports  reviewed,  the  Independent  Reference 
Review  was  completed  by  a  member  of  the  same  team  that  worked  on  the  audit  without 
an  authorizing  waiver.  AAA  SAP  auditors  stated  that  they  were  unable  to  provide  an 
auditor  who  was  independent  of  the  audit  due  to  the  limited  number  of  staff  allowed 
access  to  the  SAP  being  audited.  Auditors  at  one  site  also  stated  that  they  are  never  able 
to  provide  an  independent  auditor  for  this  reason. 

We  also  found  misleading  or  unsupported  information  in  the  final  reports.  In  one 
example,  a  figure  in  the  report  was  referred  to  as  “FY  2003”  information;  however,  the 
working  papers  documented  that  the  figure  was  “as  of  7-23-2003.”  The  information  in 
the  report  could  mislead  the  reader  since  over  two  months  of  data  in  the  report  was  not 
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supported  by  the  audit  documentation.  Another  example  is  a  change  in  the  number  of 
contractors  working  on  a  SAP.  The  draft  report  said  “one  full-time  contractor,”  and  the 
final  report  said  “seven  contractor  personnel.”  Since  there  was  no  cross-referenced 
version  of  the  final  report  (and  no  Independent  Reference  Review  of  changes  between  the 
draft  and  final  versions),  we  were  unable  to  verify  support  for  the  figure  in  the  final 
report.  In  addition,  we  found  one  report  where  the  Independent  Reference  Review  was 
not  documented  as  being  completed  and  the  referencer  did  not  place  their  initials  next  to 
each  fact  and  figure  referenced  on  the  draft  and  the  supporting  working  papers  as  required 
by  AAA  policies. 

Recommendation.  The  Army  Auditor  General  should  stress  to  SAP  auditors  the 
importance  of  independent  reference  reviews  by  personnel  that  are  independent  of  the 
audit  work,  and  ensure  compliance  with  the  requirement  to  obtain  a  waiver  and  document 
specific  extenuating  circumstances  as  to  why  independent  referencing  cannot  be 
accomplished. 

Management  Comments.  The  Army  Auditor  General  concurred  with  the 
recommendation  and  stated  that  AAA  would  issue  a  memorandum  to  all  agency  personnel 
discussing  the  results  of  the  peer  review  and  reminding  them  of  the  existing  requirements 
relating  to  the  selection  of  the  referencer.  In  addition,  the  Army  Auditor  General  stated 
that  in  instances  where  an  individual  not  directly  associated  with  the  audit  is  not  available 
to  conduct  the  Independent  Reference  Review,  the  Audit  Manager  will  document  the 
specific  circumstances  and  submit  a  request  for  waiver  to  the  Program  Director.  The 
Army  Auditor  General  stressed  that  waivers  will  be  the  exception  rather  than  the  norm  for 
all  on-going  and  future  audits. 

Reviewer  Response.  Management  comments  are  responsive.  The  AAA 
memorandum  discussing  the  results  of  the  peer  review  and  reminding  agency  personnel 
of  the  existing  requirements  relating  to  the  selection  of  the  referencer  was  issued  on 
July  29,  2005. 

Quality  Assurance.  GAS  requires  each  organization  to  have  an  appropriate  internal 
quality  control  system  in  place.  As  part  of  the  AAA  quality  control  program,  AAA 
Regulation  36-3  requires  a  quality  control  checklist  be  completed.  The  checklist  includes 
steps  for  performance  requirements,  standards,  and  auditor  independence;  entrance  and 
exit  conferences;  reviewing  prior  reports  and  gathering  background  data;  audit  programs; 
data  gathering  and  analysis;  due  professional  care;  illegal  acts  and  abuses;  working 
papers;  Command  relations;  and  supervisory  controls.  Since  the  need  for  special  access 
and  security  requirements  in  a  SAP  environment  can  impact  the  extent  of  supervision, 
documentation,  and  quality  controls  compared  to  an  unclassified  environment,  this 
checklist  serves  as  an  important  procedure  for  ensuring  quality  in  SAP  audits. 

For  the  three  audits  reviewed,  the  AAA  SAP  auditors  prepared  the  quality  control 
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checklist;  however,  we  found  no  explanations  for  management  items  that  were  not 
completed  or  omitted,  and  the  Program  Director  did  not  always  sign  the  checklist. 

GAS  3.50  requires  that  an  audit  organizations  internal  quality  control  system  should 
include  procedures  for  monitoring,  on  an  ongoing  basis,  whether  the  policies  and 
procedures  related  to  the  standards  are  suitably  designed  and  are  being  effectively 
applied.  This  often  referred  to  as  an  internal  quality  assurance  program. 

AAA  Regulation  36-62,  “Quality  Assurance  Program  and  External  Quality  Control 
Review  Responsibilities,”  provides  guidance  for  conducting  the  quality  assurance 
program  within  the  AAA.  The  regulation  states  that  AAA  Quality  Assurance  Branch 
personnel  will  make  periodic  internal  quality  control  reviews  of  selected  audit  projects. 
The  reviews  will  be  structured  to  evaluate  compliance  with  GAS.  Depending  on  the 
priorities  established  by  the  Auditor  General,  the  3-year  plan  will  include  a  mix  of  audit 
and  non-audit  projects.  The  regulation  further  states  that  all  AAA  activities  and  functions 
are  subject  to  internal  quality  control  reviews  except  for  SAPs.  However,  SAPs  are 
subject  to  external  quality  control  reviews. 

AAA’s  management  comments  to  the  original  draft  of  this  report  indicated  that  it  was 
AAA’s  intent  to  include  the  SAP  audits  in  their  internal  quality  assurance  review 
program.  AAA  should  modify  AAA  Regulation  36-62  to  specifically  include  SAP  audits 
as  part  of  the  internal  quality  assurance  program.  We  recognize  that  the  usual  AAA 
quality  assurance  program  may  need  to  be  adapted  to  the  SAP  environment;  therefore,  the 
regulation  should  address  the  SAP  environment.  Having  the  internal  quality  assurance 
visibility  within  the  SAP  environment  is  critical  to  an  effective  quality  program  given  the 
challenges  faced  in  meeting  the  other  quality  processes  identified  in  the  report  and  given 
the  rare  occurrence  of  an  external  peer  review  within  the  environment. 

Recommendation.  The  Army  Auditor  General  should  remind  SAP  auditors  of 
existing  requirements  to  complete  a  quality  control  checklist,  including  explaining  all 
omitted  steps,  and  obtaining  appropriate  Program  Director  signature  and  date  for  the 
completed  checklist. 

Management  Comments.  The  Army  Auditor  General  concurred  with  the 
recommendation  and  stated  that  AAA  would  issue  a  memorandum  to  all  agency  personnel 
discussing  the  results  of  the  peer  review  and  reminding  them  of  the  existing  requirements 
to  complete  the  Quality  Control  Checklist  and  obtain  the  appropriate  Program  Director’s 
signature  and  date  for  the  checklist.  In  addition,  the  Army  Auditor  General  stated  that  by 
September  30,  2005,  AAA  will  include  a  standard  policy  paragraph  on  the  Quality 
Control  Checklist  in  the  regulations.  The  policy  will  direct  auditors  to  find  the  most 
current  checklist  and  to  provide  an  explanation  for  any  omitted  checklist  steps. 

Reviewer  Response.  Management  comments  are  responsive.  The  AAA 
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memorandum  discussing  the  results  of  the  peer  review  and  reminding  agency  personnel 
of  the  existing  requirements  to  complete  the  Quality  Control  Checklist  and  obtain  the 
appropriate  signature’s  on  the  checklist  was  issued  on  July  29, 2005. 

Recommendation.  The  Army  Auditor  General  should  amend  AAA  Regulation 
36-62  to  eliminate  the  exception  for  SAPs  and  if  needed,  adapt  the  internal  quality 
assurance  procedures  for  the  SAP  environment. 

Management  Comments.  The  Army  Auditor  General  concurred  with  the 
recommendation  and  stated  that  by  October  31,  2005,  AAA  will  issue  an  amended 
AAA  Regulation  36-62  that  will  include  removal  of  the  exception  for  SAP  audits  in  the 
AAA  internal  quality  assurance  procedures. 

Reviewer  Response.  Management  comments  are  responsive. 
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Appendix  C.  Management  Comments 


DEPARTMENT  OF  THE  ARMY 

OFFICE  OF  THE  AUDFTOR  GENERAL 
3101  PARK  CENTER  DRIVE 
ALEXANDRIA,  VIRGINIA  22303-1596 


SAAG-ZA  1  July  2005 


MEMORANDUM  FOR  Assistant  Inspector  General,  Audit  Policy  and  Oversight, 
Inspector  General,  Department  of  Defense,  400  Army  Navy  Drive,  Arlington,  VA 
22202-4704 

SUBJECT:  Quality  Control  Review  of  Special  Access  Program  Audits 


1 .  We  are  providing  our  written  response  (enclosure)  to  the  draft 
recommendations  in  Appendix  B  to  your  memorandum  dated  15  June  2005.  We 
concur  with  the  intent  of  all  of  the  recommendations  in  Appendix  B,  Comments, 
Observations,  and  Recommendations. 

2.  We  appreciate  the  timeliness  of  your  opinion  report,  and  the  opportunity  to 
provide  comments  pertaining  to  your  draft.  We  are  especially  pleased  with  your 
conclusion  that  our  system  of  quality  control  was  designed  in  accordance  with  quality 
standards  established  by  government  auditing  standards  and  provides  reasonable 
assurance  that  our  auditors  working  on  Special  Access  Programs  are  following 
established  policies,  procedures,  and  applicable  auditing  standards. 

3.  We  thank  you  and  your  staff  for  their  professionalism  in  performing  the  quality 
control  review  and  bringing  to  our  attention  areas  where  we  can  improve  our 
operations.  If  you  have  any  questions  or  need  additional  information,  please  call 
Ms.  Theresa  Wilson  at  (703)  428-721 3  or  e-mail  Theresa.Wilson@aaa.armv.mil  or 
Ms.  Lynn  Houck  at  (703)  428-7222  or  e-mail  Patricia.Houck@aaa.armv.mil. 

Enel  JQYCEE.  MORROW 

Tne  Auditor  General 


Comments  on  Appendix  B.  Comments,  Observations,  and  Recommendations 
Quality  Control  Review  of  Special  Access  Program  Audits 
(Project  No.  D2005-DIP0AJ-0098) 


Overall:  We  are  pleased  with  your  overall  conclusion  that  the  U.S.  Army  Audit 
Agency  (USAAA)  has  implemented  a  comprehensive  internal  quality  control  program 
that  complies  with  government  auditing  standards,  is  operating  effectively,  and 
provides  reasonable  assurance  that  audit  teams  follow  internal  policies  and 
procedures  and  applicable  government  auditing  standards.  Our  written  comments 
pertain  to  the  recommendations  in  the  draft  memorandum,  Quality  Control  Review  of 
the  Army  Audit  Agency’s  Special  Access  Program  Audits, 


Supervision 

Recommendation:  We  recommend  that  the  Army  Auditor  General  develop 
alternative  policy  guidance  or  amend  AAA  Regulation  36-3  to  include  procedures  for 
documenting  supervisory  review  of  the  work  of  AlCs  within  the  SAP  environment. 

USAAA  Comments:  We  concur  that  supervisory  reviews  must  be  performed  and 
documented;  however,  we  disagree  that  alternative  policy  guidance  or  an 
amendment  to  USAAAR  36-3  (Audit  Survey  and  Execution)  is  necessary. 

USAAAR  36-3  states  'Supervisory  reviews  of  working  papers  are  mandatory,  and 
supervisors  must  document  their  reviews.”  This  is  applicable  to  all  auditors  whether 
they  are  working  classified,  unclassified  or  special  access  program  audits.  We 
concur  that  appropriate  supervisory  review  of  classified  documents  is  important  as  a 
quality  control  measure. 

During  the  course  of  this  review,  the  Program  Director  for  SAP  audits  issued 
guidance  emphasizing  the  requirement  for  supervisory  reviews.  By  1 5  July  2005, 
the  Deputy  Auditor  General,  Policy  and  Operations  Management  will  issue  a 
memorandum  to  all  Agency  personnel  discussing  the  results  of  the  peer  review  of 
the  Agency  and  reminding  Agency  personnel  of  the  requirements  in  USAAAR  36-3 
relating  to  supervisory  review  of  working  papers  and  documenting  those  reviews. 
Supervisory  review  of  working  papers  will  continue  to  be  a  special  interest  item  on 
future  quality  assurance  post  audit  reviews  and  on  FY  06  functional  reviews.  In 
addition,  our  new  Independent  Review  Checklist  includes  a  question  on  whether  the 
working  papers  have  been  reviewed-  The  independent  reviewer  must  comment  on 
whether  the  working  papers  have  been  reviewed.  The  Independent  Review 
Checklist  will  be  included  in  USAAAR  36-4  (Report  Writing  Process).  We  will  issue 
USAAAR  36-4  by  30  September  2005. 


Enclosure 
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Evidence  and  Audit  Documentation 

Recommendation:  The  Army  Auditor  General  should  remind  all  SAP  audit 
personnel  to  comply  with  established  guidance  for  working  papers,  documentation, 
and  audit  evidence. 

USAAA  Comments:  Concur.  By  15  July  2005,  the  Deputy  Auditor  General,  Policy 
and  Operations  Management  will  issue  a  memorandum  to  all  Agency  personnel 
discussing  the  results  of  the  peer  review  of  the  Agency,  and  reminding  Agency 
personnel  of  the  established  guidance  for  working  paper  preparation  and  content. 

All  working  papers  will  be  appropriately  marked  with  purpose,  source,  scope,  results, 
and  conclusion.  The  memorandum  will  remind  Agency  personnel  of  the 
requirements  in  USAAAR  36-3  relating  to  documenting  audit  work  and  cross 
referencing  the  work. 


Reporting 

Recommendation:  The  Army  Auditor  General  should  stress  to  SAP  auditors  the 
importance  of  independent  reference  reviews  by  personnel  that  are  independent  of 
the  audit  work  and  ensure  compliance  with  the  requirement  to  obtain  a  waiver  and 
document  specific  extenuating  circumstances  as  to  why  independent  referencing 
cannot  be  accomplished. 

USAAA  Comments:  Concur.  By  15  July  2005,  the  Deputy  Auditor  General,  Policy 
and  Operations  Management  will  issue  a  memorandum  to  all  Agency  personnel 
discussing  the  results  of  the  peer  review  of  the  Agency  and  reminding  Agency 
personnel  of  the  requirements  in  USAAAR  36-85  (Independent  Report  Referencing) 
relating  to  selection  of  a  referencer. 

All  efforts  will  be  made  to  ensure  that  the  independent  referencer  is  not  directly 
associated  with  the  special  access  program  audit  under  review.  In  instances  where 
this  is  not  possible,  the  Audit  Manager  will  document  the  specific  circumstances  and 
submit  a  request  for  waiver  to  the  Program  Director  for  approval  as  prescribed  by 
USAAAR  36-85.  However,  waivers  for  an  independent  referencer  will  be  the 
exception  rather  than  the  none  for  all  on-going  and  future  audits. 


Quality  Assurance 

Recommendation:  The  Army  Auditor  General  should  remind  SAP  auditors  of 
existing  requirements  to  complete  a  quality  control  checklist,  including  explaining  all 
omitted  steps,  and  obtaining  appropriate  Program  Director  signature  and  date  for  the 
completed  checklist. 


Enclosure 
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USAAA  Comments:  Concur.  By  15  July  2005,  the  Deputy  Auditor  General,  Policy 
and  Operations  Management  will  issue  a  memorandum  to  all  Agency  personnel 
discussing  the  results  of  the  peer  review  of  the  Agency  and  reminding  Agency 
personnel  of  the  requirements  to  complete  the  Quality  Control  Checklist  and  obtain 
the  appropriate  program  director's  signature  and  date  for  the  checklist.  The 
memorandum  will  inform  Agency  personnel  that  the  checklist  dated 
1 2  February  2004  is  the  checklist  that  they  should  be  using.  This  version  of  the 
checklist  is  currently  available  in  USAAAR  36-62.  The  checklist  can  also  be 
obtained  from  the  audit  templates  file  and  the  quality  assurance  Web  page  on  the 
Agency’s  intranet.  The  various  sections  of  the  checklist  and  completing  the  checklist 
are  discussed  in  USAAAR  36-2  (Guidance  for  Planning  an  Audit  Engagement), 
USAAAR  36-3  (Audit  Survey  and  Execution),  Draft  USAAAR  36-4  (Report  Writing 
Process),  and  USAAAR  36-5  (Report  Reply  and  Follow-up  Process).  By 
30  September  2005,  we  will  include  a  standard  policy  paragraph  on  the  Quality 
Control  Checklist  in  the  regulations.  The  policy  will  direct  auditors  to  find  the  most 
current  checklist  in  the  templates  file  On  the  intranet.  The  updated  Quality  Control 
Checklist  will  require  an  explanation  for  omitted  checklist  steps. 


Enclosure 
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DEPARTMENT  OF  THE  ARMY 

U.$.  ARMY  AUDIT  AflENCY 

OFFICE  OF  THE  PIUNOTAL  DEPUTY  AUDITOR  QEHERAL 
3101  PARK  CENTER  DRIVE 
ALEXANDRIA,  VA  22302-1 3M 


19  August  2005 


MEMORANDUM  FOR  Assistant  Inspector  General,  Audit  Policy  and  Oversight,  Inspector 
General,  Department  of  Defense,  400  Army  Navy  Drive,  Arlington,  VA  22202-4704 

SUBJECT:  Quality  Control  Review  of  Army  Audit  Agency’s  Special  Access  Program  (SAP) 
Audits  (Project  No.  D2005-DIPOAI-0098) 


1 .  Here  is  our  written  response  to  your  draft  report  dated  1  August  2005.  We  are 
providing  comments  to  the  recommendation  added  on  page  10  relating  to  excluding  Agency 
SAP  audits  from  periodic  internal  quality  assurance  reviews.  We  choose  not  to  respond 
again  to  the  other  recommendations.  We  concur  with  the  intent  of  all  of  the 
recommendations  in  Appendix  B,  Comments,  Observations,  and  Recommendations.  The 
added  recommendation  and  our  reply  follow. 

Recommendation:  The  Auditor  General  should  amend  AAA  Regulation  36-132  to 
eliminate  the  exception  for  SAP  audits  and  if  needed,  adapt  the  internal  quality  assurance 
procedures  for  the  SAP  environment. 

USAAA  Reply:  Concur.  In  our  past  internal  quality  assurance  reviews,  we  included 
non-SAP  audit  reports  completed  by  the  team  that  performs  the  SAP  audits.  Also  the  SAP 
audits  are  subject  to  external  peer  review.  However,  we  agree  that  we  should  not  exclude 
any  audit  or  group  of  audits  from  selection  for  internal  quality  assurance  review.  We  are  in 
the  process  of  updating  USAAAR  36-62.  We  are  planning  to  complete  the  update  by 
31  October  2005.  We  will  amend  the  regulation  to  temove  the  exception  for  SAP  audits. 
Because  there  is  no  exception  for  SAP  audits,  they  will  be  covered  by  our  quality  assurance 
procedures. 

2.  We  appreciate  the  timeliness  of  your  opinion  report,  and  the  opportunity  to  provide 
comments  pertaining  to  your  draft.  We  are  especially  pleased  with  your  conclusion  that  our 
system  of  quality  control  was  designed  in  accordance  with  quality  standards  established  by 
the  Government  Auditing  Standards  and  provides  reasonable  assurance  that  our  auditors 
working  on  special  access  programs  am  following  established  policies,  procedures,  and 
applicable  auditing  standards- 
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SAAG-PMZ 

SUBJECT:  Quality  Control  Review  of  Army  Audit  Agency’s  Special  Access 
Program  Audits  (Project  No.  D2005-DIPOAI-0098) 


3.  Wc  thank  you  and  your  staff  for  their  professionalism  in  performing  the  quality  control 
review  and  bringing  to  our  attention  areas  where  we  can  improve  our  operations.  If  you 
have  any  questions  please  call  Mr.  Charles  T.  Lyons  at  703-681-8391  or  email 
eharles.lyons@aaa.army.mil  or  Ms.  Jo  L.  Spielvogel  at  703-681-4288  or  e-mail 
jo.spielvogel@aaa.artny.mil. 


JPSEPH  P.  MIZZCMT 
■'Deputy  Auditor  General 
Policy  and  Operations  Management 
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